Skip to content

server: accept agent cache hints#82

Merged
ai-hpc merged 1 commit into
GeniePod:mainfrom
bittoby:dynamo-152-runtime-agent-hints
Jun 6, 2026
Merged

server: accept agent cache hints#82
ai-hpc merged 1 commit into
GeniePod:mainfrom
bittoby:dynamo-152-runtime-agent-hints

Conversation

@bittoby

@bittoby bittoby commented May 23, 2026

Copy link
Copy Markdown
Contributor

Refs GeniePod/genie-claw#152.

Summary

  • parse top-level conversation_id plus nvext.agent_hints.session_id
  • accept priority, OSL, speculative-prefill, and ephemeral cache TTL metadata
  • sanitize session ids before using them for persistent KV sessions
  • echo accepted non-streaming hint metadata under jetson.agent_hints

Verification

  • git diff --check origin/main..HEAD
  • cmake -S . -B /tmp/genie-ai-runtime-check -DJLLM_BUILD_SERVER=ON could not complete on this Mac host because CUDA/nvcc is not installed: Failed to find nvcc.

@ai-hpc ai-hpc merged commit 7c0b259 into GeniePod:main Jun 6, 2026
@ai-hpc

ai-hpc commented Jun 6, 2026

Copy link
Copy Markdown
Member

reviewd and merged at 7c0b259

Validated on Jetson Orin Nano (built the #82#84 stack on top of #85, deployed, ran live): nvext.agent_hints parses and echoes under jetson.agent_hints (session_id sanitized, cache_control.ttl "15m" → 900s, priority/osl carried through). Clean, well-guarded parsing (clamped ints, sanitized ids). Thanks @bittoby

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants